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ABSTRACT 

This paper d< 3cusses a method of eliciting parental 
self -reports about child-rearing behavior. The method, called 
computer-presented social situations (CPSS), uses vignettes that are 
presented on computers. Several studies examined: (1) what mothers 
think about the technique; (2) reliability of data collected by the 
technique; and (3) validity of data compared to actual maternal 
behavior. In one study, mothers responded through computers, 
questionnaires, or interviews to 24 questions concerning child 
misbehavior vignettes. The CPSS was rated more favorably on four 
factors than were the other methods of eliciting self -reports, but 
were rated less favorably on expression than was the interview. A 
second study failed to replicate these results. In a third study, 
mothers responded twice, before and after a two-week interval, to a 
104-question CPSS program. Test-retest results were acceptable and 
compared favorably with other methods. In a fourth study, mothers 
were observed on a visit to a supermarket with their children , and 
responded to a CPSS program about child misbehavior during the visit 
The overall agreement between mothers 9 observed behavior and 
intentions assessed by beans of the computer was 36 percent. A 
reference list of six items is included. (BC) 
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Computer-elicited parental self-reports: Reactions to, reliability, and behavioral validity 

I'd like to talk this morning about some of what I've learned in developing and testing a new 
method for eliciting parental self-reports about their child-rearing behavior. Collecting self-reports, 
let me acknowledge from the beginning is a dirty business, but sometimes someone has to do it It 
has to be done in order to: 1) assess low frequency behaviors-such as parental use of physical 
punishment, 2) avoid infringement on a family's privacy, or 3) collect data in a limited amount of 
time, among other reasons. 

Social developmental researchers have long relied on self-reports (at least since the 1930s) and 
will continue to. The question is how can we do it well? I've previously argued that global 
attitude questionnaires are not the way to do it (Holden & Edwards, 1989). Instead, I'm 
convinced that context specific vignettes are much better (a method originally pioneered 35 years 
ago by Jackson, 1956). And, I believe the best way to present these is on microcomputers. The 
approach I've been working on I label "computer-presented social situations" (CPSS) to highlight 
the two key features, context specific situations are presented on microcomputers. The 
presentation of vignettes on computers offer a number of advantages (Table 1). 

Some of these advantages are: 1) the interactive quality of computers makes them enjoyable, 
engaging, and interesting; 2) they can be operated in private to ensure confidentiality and reduce 
the likelihood of evaluation apprehension; 3) one can readily create situationally specific vignettes 
which can be personalized with names and detailed information; 4) it allows for the examination of 
social cognition processes; and 5) computers of course have the advantages of it allows for such 
options as covert and efficient branching, systematic manipulation of variables, and automatic data 
reduction (see Roid, 1986). 

To give you an example, my first effort focused on examining a common problem solving 
process-how parents as well as non-parents diagnosing why a baby was crying. Subjects selected 
key information from the computer to evaluate which of 9 competing hypotheses was the single 
correct one (see Holden, 1988). I could then examine such variables as the types of information 
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the subjects selected, at what point in their information search they selected it, and what hypotheses 
they selected. 

Rather than talking about that work, I will focusing on how I've been using the technique more 
recently to elicit parental reports about themselves and their families. Specifically, I will be talking 
about 1) what mothers think about using the technique compared with more traditional self -report 
methods; 2) how reliable self-report data are when collected by this method; and 3) how valid the 
self report data when compared with actual maternal behavior. 

Subjective Evaluations nf the CPSS method 

The central purpose of this study was to assess mothers* reactions to the CPSS technique, in 
comparison to the two traditional approaches for eliciting self-reports-interview and questionnaire. 
Seventy mostly (73%) college-educated mothers of 3-year-old children participated. The design 
of the study was a one-way ANOVA with 3 levels of Method (Computer, Questionnaire, 
Interview). In each condition, mothers made 4 likert-type ratings to the identical 6 child 
misbehavior vignettes for a total of 24 ratings. For example, one question concerned how likely 
they would be to use three different disciplinary responses after their child ran into the street. It 
took about 20 minutes to participate. Reactions to the task were then collected on 16-item 
evaluation questionnaire. 

There was a significant multivariate effect for Method on the evaluation questions. Half of the 
10 key items were significant (Table 2). The CPSS was rated more favorably than one or both of 
the other methods as it was more enjoyable, less anxiety provoking, perceived as shorter, and 
elicited more honest responses. However, as predicted, it was rated as lower on "ability to express 
own views" than the interview. No effect of previous computer use in the evaluations was found. 

My reaction to this data was it was a good pilot study but let's get more significant effects by 
making the procedures longer. So in a second study, involving 60 mostly (63%) college-educated 
mothers of 3-year-olds, we had them do the same procedures but with more questions-a total of 66 
that were embedded in context-specific situations dealing with an average day in the life of their 
children. 
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Again, mothers were randomly assigned to one of three conditions: Computer, Survey, or 
Interview, The survey came out to be 20 pages long. Again, the identical information was 
collected in each condition. We reduced the evaluation questionnaire to include only 7 key 
questions. 

As you can see from Figure 1 (and Table 3), we found no overall significant group differences 
ana very little variation among the means. For some reason, the longer task washed out the 
differences found in Study 1. My guess is that mothers liked responding to these vignettes 
questions about their kids~whether it be on paper, interview, or computer. In conclusion, 
mothers* subjective evaluation of the CPSS method was very favorable-if not more favorable than 
other, more standard methods. 

The Reliability of CPSS Data 

A second type of study we have done to evaluate the quality of the information collected from 
the CPSS technique is to examine the test-retest reliability of the data. In this study, 20 mothers of 
3- year-olds responded to a CPSS program containing 104 questions. Then, about 2 weeks later, 
they operated the same program. (This program was used in a study with battered women [Holden 
& Ritchie, 1991]). About half were Likert-type rating scales about frequency of behavior (e.g., 
"How frequently does your child ...", probability of behaving in a particular way (e.g., "How 
likely would you be to ...), or perceptions (e.g., "How important is it that ..."). The other 
questions used a multiple choice format to assess family decisions (e.g., "Who decides what 
[child's name] will wear in the morning"), maternal reports of her disciplinary practices ("What 
would you do if [child's name] made a big mess with his/her toys"), and maternal reports of her 
husband's behavior ("What would [father's name] do if he were handling this problem?"). 

The test-retest results were very positive. With the rating scale data, intraclass correlations 
were computed (used because of the non-independent rating data) . 86% of the correlations were 
significant, with about 75% being highly significant (jj < .001). The modal correlation was .65 
(range .91 to .01). Only three out of the 47 correlations did not reveal even a trend toward a 
significant relation across the two testing times (see Figure 2). 
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With the multiple choice questions, using a conservative requirement of exact agreement across 
the two testing times, we found an average agreement of .70, (range 100 to 40%). Seventy-five 
percent was the modal agreement-indicated that on half of the questions, at least 3/4 of the mothers 
reported the exact same response on the two testing occasions. Eighty-one percent of the questions 
had exact agreement at 60% or higher (see Figure 3). 

The reason for some of the low agreements or correlations is evidently the probabilistic nature 
of parental behavior. In addition, this study clearly indicates two variables are key for the reliable 
reporting. One is the content of the question and the second is some individual difference 
characteristic of the mothers-some were much more reliable than others. We are investigating 
both of those variables further. However, the overall conclusion of this study is that the test-retest 
results of this self-report data are quite acceptable and compare favorable with other methods ~ 
such as attitude questionnaire data. 

Behavioral Validity of Self-Reports on the Computer 

The last study I will report on was designed to examine the relation between behavioral 
intentions elicited on the computer and observed behavior. Twenty-eight mothers and their 2 1/2- 
year-old children were observed during a visit to a supermarket of their choice. One week later, 
the mothers operated a CPSS program that simulated a child's misbehavior during a trip through 
the supermarket. The target questions on the computer program concerned how the mothers would 
respond to the misbehavior. Mothers then selected, in a multiple choice format, one of six basic 
responses (identified in a previous study [Holden, 1983]). 

We assessed the validity of their self reports by calculating exact agreement in the following 
procedure. First we looked at the observational data to identifying the target events of child 
misbehavior, next we identified how the mother responded in the supermarket; then we looked at 
the print-out of her response to the same type of incident on the computer simulation. If the mother 
reported that she would respond in the same way that we observed, then that was coded as an 
agreement If not, then that was coded as a disagreement 
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The overall exact agreement between mother?' observed behavioral responses and their 
behavioral intentions assessed on the computer was only 36% (Table 4). 

However, for a salient class of maternal behaviors-primarily when the mothers had to use a 
power assertive type of response-the exact correspondence was quite respectable with an average 
of 72%. But with less salient responses, such as reasoning, consenting, or ignoring a 
misber avior-the correspondence was low. Again in this study there was evidence of some 
individual differences operating as revealed by the six most accurate mothers averaged 61% of 
exact agreement across all categories. 

Conclusion 

So I want to summarize what we've learned about this methodology. First, mothers do like it 
It is unclear whether they like it better than more traditional ways, but it certainly doesn't do any 
worse, in terms of their subjective evaluations. Second, the vast majority of the questions 
responded to in this format of context specific vignettes demonstrate at least acceptable test-retest 
reliability and in some cases excellent reliability. Finally, we've demonstrated some validity to the 
self-reported behavioral intentions. In particular, those questions that deal with salient maternal 
responses appear to be quite accurately reported. 

With the technological afforded by microcomputers, progress in developing improved self- 
reported measures of parental behavior for the study of social development is on the horizon. 
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Table 1. Potential advantages of the CPSS technique 

1) Interesting, engaging, and enjoyable 

2) Confidential and anonymous— removes experimenter from situation 

3) Creation of context specific "problem space" is readily achieved 

4) Examine social cognition processes 

5) Computer allows for discreet branching, systematic manipulation of variables, automatic 
data reduction and analysis 



Table 2. Significant group differences in maternal evaluations of the methods-Study 1 



Variable 



Significance 



groups that differed 



Enjoyment 

Anxiety provoking 

Subjective sense of duration of task 

Ability to express own views 

Honesty of responses 



.06 

.05 

.01 

.001 

.01 



C>I 

I>CQ 

Q>C 

I>CQ 

C>Q 



MANOVA = EI32, 981= 2.36, fi < .001 

Key: C = CPSS, I = Interview, Q = Questionnaire 
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Table 2 (cont) Means for group differences in maternal evaluations of the methods-Study 1 



Variables Gmm - 

Computer Interview Questionnaire 

Enjoyment 6.1 (liked very 5.4 5.8 

much) 

Nervous 1.2 (Not at all) 1.8 1.3 

Duration of tosk 3.4 (Fairly short) 3.7 4.0 

Express own views 5.0 (Fairly well) 6.0 4.8 

Honesty of responses 6.8 (Completely) 6.6 6.3 



Table 3. Means for group differences in maternal evaluations of the methods-Study 2 



Variables Croups . 

Computer Interview Questionnaire 

Enjoyment 6.3 (liked very 5.8 5.8 

much) 

Nervous 1.6 (Slightly) 1.3 1.6 

Express own views 4.8 (Fairly well) 5.1 4.9 

Honesty of responses 6.2 (Very) 6.3 6.3 

Judged 2.3 (A little) 1.4 2.4 

Duration of task 3.9 (Right) 4.1 3.8 

Realistic 5.6 (Fairly) 5.7 5.7 



MANOVA = 014,961= 1.14.ji<.33 
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Table 4. Supermarket Study: Percent Exact Agreement Between the Observational and 
CPSS Data 



Quid 
Behavior. 



Reasons 



Opens 74 
Grocery 



Maternal Response 

Power Diverts Consents Ignores 
Assertion 



79 

(H/14) 



50 

(21/42) 



31 

(15/49) 



(0/6) 



Plays 

with 

Grocery 

Reaches 
for 

Grocery 



23 
(5/22) 

15 

(3/20) 



Requests 21 
Item 



(37/173) 



100 
(13/13) 

75 
(21/28) 

71 

(35/49) 



0 

(0/1) 



53 
(16/30) 



0 

(0/4) 
0 

(0/7) 

21 
(15/70) 



0 

(0/1) 
0 

(0/2) 

0 

(0/36) 



Wants 
Candy 



0 

(0/8) 



100 

(6/6) 



0 



(0/2) 



(0/8) 



Marginal 78 



57 



51 



19 



0 



Notes . A dash indicates a structural zero because that response was not available in the 
CPSS. The numbers in parentheses indicate the observed frequency of exact 
correspondence over correspondence plus non-correspondence. For example, the numbers 
"14/19" below the mean of 74 in the top left cell of the table indicate that mothers were 
observed to use power assertions as a response to their children's opening a grocery on 19 
occasions. In 14 instances, the same mothers also selected a reasoning response to that 
situation on the CPSS. 
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Study 2 - Exit Survey Results 
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Test-Retest Correlations from Rating Scales 
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Figure 3. 



Test-Retest % Exact Agreement from MC Questions 
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